TRANSFERABILITY OF COMBAT MODELS 
LIMITATIONS IMPOSED BY 
DOCUMENTATION PRACTICES 



Robert Walter Szymczak 



NAVAL POSTGRADUATE SCHOOL 

Monterey, California 




THESIS 



TRANSFERABILITY OF COMBAT MODELS » 
LIMITATIONS IMPOSED BY 
DOCUMENTATION PRACTICES 

Robert '.Valter Szymczak 
September 1979 



Thesis Advisors J. G, Taylor 

Approved for public release; distribution 
unlimited 



T 191974 




UNCLASSIFIED 



SECURITY CLASSIFICATION OF THIS PACE (Whan Dmtm Entmrad) 



REPORT DOCUMENTATION PAGE 


READ INSTRUCTIONS 
BEFORE COMPLETING FORM 


1. REPORT NUMBER 


2. GOVT ACCESSION NO. 


1. RECIPIENT’S CATALOG NUMBER 


4. TITLE (ond Subtltlo) 

Transferability of Combat Models; 
Limitations Imposed by 
Documentation Practices 


5. TYPE OF REPORT * PERIOO COVERED 

Master's Thesis: 
September 1979 


S. PERFORMING ORG. REPORT NUMBER 


7. AUTHOR^*) 

Robert Walter Szymczak 


i. CONTRACT OR GRANT NUMBERfa) 


». PERFORMING ORGANIZATION NAME ANO AOORESS 

Naval Postgraduate School 
Monterey, California 939^-0 


10. program element, project, task 

AREA * WORK UNIT NUMBERS 


II. CONTROLLING OFFICE NAME ANO AOORESS 

Naval Postgraduate School 
Monterey, California 939^0 


12. REPORT OATE 

September 1979 


11. NUMBER PAGES 


14. MONITORING AGENCY NAME * AOORESVK Irom Controlling Ollleo) 

Naval Postgraduate School 
Monterey, California 939^0 


IS. SECURITY CLASS, (o 1 thl, report) 

Unclassified 


U«. OECL ASSIFI CATION/ DOWNGRADING 
SCHEDULE 


l«. DISTRIBUTION STATEMENT ( ot 1>l* Koport) 

Approved for public release; distribution unlimited. 


17. DISTRIBUTION STATEMENT (of thm mbmtrmct mntmrmd In Stack 20, it diiimrmnt tram Rmpart) 


it. supplementary notes 


19. KEY WORDS (Continue on rmwmrmm midm it nmcmmmmry and idmntity by block tummbmr) 

Combat Models Theatre-Level Model 

Model Documentation Transparency 

Large-Scale Model Transferability 


20. ABSTRACT (Conttnum an rarer mm midm it nmemmmmry and idmntity by btaek number) 

This thesis proposes a hierarchy of documentation for 
combat models. It begins by examining criticisms and 
credibility of combat models to establish underlying causes 
and effects, and then it addresses model proliferation and 
ever increasing complexity as they affect one’s ability to 
understand and transfer models. A methodology for determing 
whether or not a model is applicable to a specific problem 



DO ,j2J M 7J 1473 COITION OF 1 NOV IIIIOISOLITI UNCLASSIFIED 

(Page 1) S/N 0 102 * 0 14* 660 1 | 



SECURITY CLASSIFICATION of THIS P AGE (Whmn Dmtm Kntmrmd) 



UNCLASSIFIED 

f#CUWJTV CL A TIQN O' This 



n*i« trnt* 



a 



is presented, as are examples of potential problem 
areas. Current documentation practices are examined 
for conditions that limit the transferability of 
models and contribute to the credibility problem. 

The above examinations have lead to a proposed 
three-tier hierarchy of documentation, including for 
the analyst documentation that is presented from the 
context of discovery rather than from the traditional 
context of justification. Recommendations are made 
for supplemental studies to examine related issues. 



DD Form 1473 




UNCLASSIFIED 



2 



SfCUftITV CLAMIFICATIOK O' TMIf Qmtm 



Approved for public release; distribution unlimited 



TRANSFERABILITY OF COMBAT MODELS: LIMITATIONS IMPOSED BY 

DOCUMENTATION PRACTICES 



by 



Robert Walter £zymczalc 
Lieutenant Colonel, United States Army 
B.S., United States Military Academy, 1962 



Submitted in partial 
requirements for 



fulfillment of the 
the degree of 



MASTER OF SCIENCE IN OPERATIONS 



RESEARCH 



from the 



NAVAL POSTGRADUATE SCHOOL 
September 1979 



DUDLEY KNOX LIBRARY 
fiAVAl POSTGRADUATE SCHOOL 
MONTEREY, CA 93940 



ABSTRACT 



for 

and credibility 
underlying caus 
model prolifer 
they affect one 
models. A met 
a model is a 
presented, as 
areas. Currant 
for conditions 
models and cont 
The aoove 
three-tier hi 
the analyst 
context of di 
context of justification, 
for supplemental studies to 



of documentation 
xamining criticisms 
dels to establish 
then it addresses 
asing complexity as 
rand and transfer 
ning whether or not 
ecific problem is 
potential problem 
ices are examined 
transferability of 
dibility problem. 

sed 
for 
the 
nal 

Recommendations are made 
examine related issues. 



This thesis proposes a hierarchy 
combat models. It begins by e 

of combat mo 
es and affects, and 
ation and ever incre 
's ability to unders 
hodology for determi 
pplicable to a sp 
are examples of 
documentation pract 
that limit the 
ribute to the ere 
examinations have lead 
erarchy of documentation, 
documentation that is presented from 
scovery rather than from the traditio 



to a propo 
including 



4 



TABLE OF CONTENTS 



I. BACKGROUND AND INTRODUCTION 7 

A. BACKGROUND 7 

B. DEFINITIONS 8 

C. MODELING AND MODELS 9 

D. INTRODUCTION 10 

E. SUMMARY 12 

II. CRITICISMS AND CREDIBILITY OF COMBAT MODELS 13 

A. CAUSES OF CRITICISM 13 

B. IMPORTANCE OF ASSUMPTIONS 16 

C. MODEL EVALUATION 18 

D. SUMMARY 20 

III. PROLIFERATION, COMPLEXITY, AND TRANSPARENCY 21 

A. PROLIFERATION AND HOW TO REDUCE IT 22 

B. THE EFFECTS OF COMPLEXITY ON TRANSPARENCY.... 25 

C. SUMMARY 27 

IV. REQUIREMENTS TO TRANSFER A MODEL 29 

A. METHODOLOGY FOR ANALYSIS OF EXISTING MODELS.. 29 

B. POTENTIAL PROBLEMS IN EXISTING MODELS 32 

C. SUMMARY 36 

V. STATUS AND ADEQUACY OF CURRENT DOCUMENTATION 38 

A. DEFINITIONS AND BACKGROUND 38 

B. CURRENT DOCUMEN T ATION PRACTICES 39 

C. SUMMARY 43 

VI. PROPOSED HIERARCHY OF DOCUMENTATION 45 

A. THE USER AND THE ANALYST 45 

B. ORIGIN OF THE DOCUMENTATION SHORTFALL 49 

C. COMMUNICATION AND LEVELS OF PARTICIPATION.... 51 

D. A CONCEPTUAL THEORY OF MODEL DOCUMENTATION... 53 

E. REQUIRED LEVELS OF DOCUMENTATION 56 

F. DECISION MAKER'S NON-TECHNI CAL DOCUMENTATION. 58 



5 



G. ANALYST'S C ONCE PT OAL -TE CHNI CAL DOCUMENTATION. 59 

H. PROGRAMMER'S TECHNICAL DOCUMENTATION 62 

I. INLINE DOCUMENTATION, A MAP THRU THE MAZE 64 

J. UPDATING A MODEL'S DOCUMENTATION 65 

K. SUMMARY 66 

VII. CONCLUSIONS, RECOMMENDATIONS, AND FINAL REMARKS.. 68 

A. SUMMARY OF PROBLEMS DISCUSSED 68 

B. CONCLUSIONS AND RECOMMENDATIONS 70 

C. FINAL REMARKS 73 

LIST OF REFERENCES 77 

INITIAL DISTRIBUTION LIST 81 



6 



I. 



BACKGROUND AND INTRODUCTION 



A. BACKGROUND 



Motivation for this thesis was initially provided by 
Professor James G. Taylor, who wanted to use an existing 
fully operating large-scale combat model as a teaching 
vehicle in a com bat- modell ing course in the Operations 
Analysis Curriculum at the Naval Postgraduate School (NPS) . 
The original plan was to acquire the ATLAS (A Tactical, 
Logistical, and Air Simulation) model from the U. S. Army's 
Concepts Analysis Agency, convert it for use on the NPS IBM 
360 computer and develop a manual for the setup and running 
of the model as part of the course. Then the attrition and 
movement routines of the model were to be analyzed as a 
formal thesis. 

The project was especially appealing to the author 
because it conformed with a fundamental belief of his, that 
rather than developing new models if an existing model is 
appropriate for a given analysis it should be acquired and 
used. The concept is not original but was based on 

recommendat ions of the Army Models Review Committee. [1] 
Acquisition and use of ATLAS at the NPS would be an 
application of the recommended concept of transferring an 
existing model whenever its use is feasible. 

In late Feb. 1979, the ATLAS model arrived via a 
magnetic tape, and the author proceeded to execute the above 
plan. After the expenditure of about four man-months of 
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effort by the author and computer center programmers and 190 
minutes of CPO time, AT IAS was successfully compiled and 
linked. This expenditure of effort was greatly in excess 

of the expected time to complete the task, given that the 
model has been in existence for ten or more years and is 
considered to be "simple" compared with other theatre-level 
models. Why had its transfer required such expenditure of 
effort? 

In reflecting upon the above question, the author 
realized that even though a model may have been used for 
many years, its transfer can still be hampered by limited 
documentation. This thesis will relate the author's 
experience with ATLAS and his subsequent investigation of 
how documentation limits the transfer of existing models 
between agencies. It will also examine the related 
problems of model complexity, proliferation, credibility, 
transparency, and transferability. Based on this 
examination and the experiences of the author during seven 
years of operation research related assignments, a new 
concept of documentation to improve the transferability of 
combat models is proposed. 



3. DEFINITIONS 

In research conducted for this thesis the author 
discovered that in modeling the same word can have many 
connotations. For purposes of clarity the author adopted 
the definitions of modeling terras listed in the glossary of 
Ref. 1. This does not imply that they are the only or best 
definitions, they are only used as a point of departure. 

Within the following text there will be references to 
combat models, theatre-level, large-scale, and small-scale 
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models. For purposes of the remarks, discussions, and 
recommendations contained herein, the words can be 
considered synonymous. Since the initial impetus for the 
thesis concerned work with a theatre-level model this term 
readily crept into the author's dicussion. However, all 
these forms of models are just subsets of the concept of a 
combat modal. What is addressed throughout the thesis is 
combat modeling. Particular comments referring to one of 
the subsets are just as applicable to combat models as a 
whole . 

Although some remarks are addressed to the DOD, others 
to the Department of the Array, each is applicable to the 
other as well as the other services. The discussions, 

conclusions, and recommendations are equally applicable to 
other complex models as well as models in general. 



C. MODELING AND MODELS 



What is modeling? According to Morris [2] modeling is 
an intuitive process through which an analyst arrives at a 
model. On the other hand, a model is an inanimate object, 
an abstraction of reality [3], that is used by the analyst 
to answer questions about some future state of a process. 
This differentiation between modeling and a model is 
necessary to understand the implications of the facts 
presented and the conclusions drawn in .this thesis. 

To develop a model, the modeler goes through a process 
of discovery. This process is the trial and error procedure 
in which the designer tries to abstract the key elements of 
reality. Once he has done this and validated his model, the 
logical reconstruction of events leading to the model are 
documented in the context of justification. This logical 
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reconstruction has little if anything to do with the actual 
process followed in building the model. No attempt is made 
to verbalize the actual psychological process, the problems 
encountered, or dead ends pursued. 

Over time the initial simple model proceeds through a 
process of enrichment or elaboration. Through this process 
the model is modified and moved in evolutionary fashion 
toward a more elaborate representation of reality; in the 
process the model becomes more complex as it seeks to 
reflect the complexity of the reality it represents. Each 
time the model is enriched the reasons why and how it is 
enriched should be documented. For a discussion of the 
modeling process see Ref. 2. 



D. INTRODUCTION 



Since ancient times military planners have used wargames 
to investigate various aspects of possible future military 
operations. Historical development of war games can be 
found in Ref. 4 or other histories of war games in the open 
literature. The oldest known form of the wargame is a 
Hindu chess-like game called "Chaturanga". Modern war 
games had their beginning in 1664 thru the games developed 
by Christopher Werkmann called military chess. In the 
twentieth century the greatest proponent of war games till 
the end of WWII was the German Army. Through the 
development of the digital computer during the war a new 
facet of wargaming became possible. The use of the computer 
greatly reduced the time and effort required to conduct a 
war game. Computer assisted or computer run war games 

provide a means of gaining insights and experience in 
military problem solutions. [5] These computer war games 
help evaluate new weapons systems, study current and 
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proposed military organizations and investigate the possible 
outcomes of future conflicts given particular weapons, 
organizations, tactics, and enemy forces. The basis of such 
computer wargames is the combat model. Although there are 
many military applications of combat models, this thesis is 
primarily concerned with those applicable to military 
strategy and force planning. 

Military strategy and force planning has matured since 
WWII. Prior to WWII military technology evolved slowly; 

those responsible for strategy and planning could easily 
gain all the necessary knowledge about the relationship of 
military forces and weapon systems to national security from 
books or personal experience. Within one lifetime the 
amount of technological change was not sufficient to render 
experience invalid. The military did not plan on 

technological change; it merely adjusted to it. 

During and subsequent to WWII, the rate of change of 
military technology began to increase in an almost 
exponential manner. A lifetime of experience could became 
obsolete in a few years; now mere adjustment to change was 
not sufficient, the military had to plan for change. This 
revolution in technology was incubated and nurtured by the 
advent of the digital computer: to keep pace with 

technology and its impact on strategy and force planning the 
digital computer was adopted as a planning tool. With it 
the means to assess and adapt technological change and 
incorporate it into military strategy and force planning was 
possible . [ 6 ] 

Modern day force planning has become largely an 
analytical process that necessarily employs the digital 
computer. The digital computer has been incorporated into 
many of the myriad aspects of military planning and decision 
making in order to provide a scientific basis to these 
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activities. In this thesis, the computer is considered only 
as a computational aid for combat models. The outputs of 
these models are used as aids to decision makers. Strategy 
and force planning depend primarily on large-scale combat 
models, which due to their complexity require the use of 
digital computers. Models can be of various types or 
classes; there is no agreed upon system of 
classification. Discussions of classification of combat 
models are contained in the literature. [3,7 8] This study 
is concerned with large scale computer simulations and 
analytical models as defined in Ref. 3. 



2. SUMMARY 



This chapter has discussed the background of this 
thesis, and it has provided some basic definitions and a 
general history of combat modeling to the 

present. Subsequent chapters will now address the following 
important aspects of models used for defense planning: 
criticism and credibility; proliferation, complexity and 
transparency, and requirements to transfer a model. Next 
a common element of each chapter, lack of adequate 

documentation, is identified as a contributing cause to each 
of the conditions discussed. This is followed by a 

proposed concept of documentation that will alleviate many 
of the problems discussed in this thesis. 
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II 



• CRITICISES AND CREDIBILITY OF COMBAT MODELING 



A. CAOSES OF CRITICISM 

Critics and criticism of defense analysis and its tools, 
of which modeling is just one, are ever present. Criticism 
has not abated since the early sixties; it continues to the 
present and threatens funding, the life blood of analysis, 
in the Department of Defense. Informed private citizens and 
activitists groups have attacked the propriety of defense 
analysis and DOD decisions based on analysis. The poor 
public image was cited as a contributing factor to the poor 
reception of analysis in Congress. Because of seeming 
inconsistencies in analysis. Congress has become skeptical 
and disenchanted and has questioned the utility of 
analysis . [ 9 ] 



According to surveys an 
decade [10] activity aad 
simulation peaked in the m 
decline since then. These 
machine simulation had ge 
time Operations Research a 
critical self-examination, 
and encompassed a wide 
relationships. Those of 
about the inter-related 
complexity, proliferation a 



d studies conducted early in this 
expenditures on gaming and 
iddle 1960's and were on a slight 
investigations indicated that 
nerally been oversold and at that 
nd modeling were undergoing a 
[11] The criticisms were many 
variety of cause and effect 
relevance to this study revolve 
areas of transferability, 
nd documentation. 
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As the use of combat models increased, so did their 
initial acceptance and importance, and in turn the 

complexity of these models has also increased. Complexity 
has manifested itself in various forms: inputs, types of 

models, language, detail of actions and conditions 

simulated, simulated decision making and computing 

machines. Eventually the complexity of existing models, 
poor transparency, and difficulty of transferability caused 
numerous models to be developed that modeled the same or 
very similar combat phenonema. Unfortunately, at this 
juncture in the development of combat models (late 60's, 
early 70's) criticism and dwindling credibility occurred. 
This came about because for reasons not easily recognizable 
or understandable to decision makers, models supposedly 
modeling the same combat process under the "same" conditions 
produced different and at time conflicting results. 

Analysis implies rigor and association with the 
scientific method, yet standards seemed to have waned; 
strict adherance to standards of scientific rigor and 
discipline were less than tenacious. Often analyses and 
models produced had methodological flaws. Often these flaws 
were not discovered until after an analysis had been 
accepted and decisions made. In studies involving models, 
one of the contributing factors to this situation was lack 
of detailed understanding of the model. Other contributing 
factors were the pressure of time and limited 
distribution. In the rush to meet deadline the guality of 
the work was often sacrificed. By not distributing a study 
or model to other agencies the extra set of eyes that can 
see a fatal flaw through an unbiased view were never used. 

The use of more than one model has often resulted in 
decision makers being confronted by seemingly contradictory 
results of different analyses using different models 
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addressing the same problem. No wonder that they have the 
impression that the models and methods used by analysts may 
not be very objective guides. Yet most analysts will argue 
that a detailed comparison of models, their assumptions, 
inputs and calculations, show that the results are not 
really contradictory. Differences in outputs are usually 
the direct result of differences in assumptions and methods 
of processing the input data. This in turn presents the 
analyst with two challenges: First, he must recognize and 

understand these seeming contradictions, and then he must 
resolve and convey to the decision maker these 
differences. Dr. Wilbur B. Payne, former Deputy 

Undersecretary of the Army for Operations Research speaking 
from the point of view of the decision maker who tries to 
draw valid conclusions from analyses that use such models, 
has argued that he has frequently seen such apparently 
contradictory results coming from various models addressing 
essentially the same problem. Furthermore, the decision 
making process in large organizations is such that the 
detailed comparison , if it is ever done , and resolution of 
seeming conflicts usually does not reach the decision 
maker. Hence, the decision maker is left with the 

conclusion that simulation results are not consistant and 
therefore of dubious reliability. [12] 



Contradictory results of combat models is a factor cited 
by Huber [12] that has caused the very credibility of combat 
modeling to be questioned. Under ideal conditions a model 
should be directly connected with a continuing experimental 
program and should reasonably relate to other models that 
simulate the same or similar processes. The user must be 
especially watchful in this respect because the combat 
process does not easily lend itself to establishing a 
continuing experimental program. Many combat models are 
neither built nor used with any forethought given to their 
connection to other models. Each generally turns out to be 
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a totally independent data generator. This precludes any 
meaningful experimental feedback in the on-going prediction 
process and results in a mass of unrelated and often 
contradictory data generated by many models. 

The Army Models Review Committee report was the seminal 
publication on theatre-level combat models for the 
seventies. From it sprang a decade of proposals and counter 
proposals concerning theatre-level combat models. However, 
if one scrutenizes the literature of the previous decade a 
feeling of deja vu surfaces. Many of the ideas and 
arguments sounded in the seventies are in the literature of 
the sixties. For an example of the problems foreseen and 
warnings given, see Ref. 13. 



3. IMPORTANCE OF ASSUMPTIONS 



One of the key facets of any model is the assumptions 
that go into its development. The importance of 

assumptions w-hether in models or otherwise was recognized 
early in the development of systems analysis. When Me 

Namara was Secretary of Defense, a continuing effort was 
made to insure assumptions incorporated into models were 
both explicit and consistent. Whether comparing force 

structure or strategy. Me Namara considered it possible to 
select assumptions that will make any proposed weapon system 
or organization look optimal. [6] Likewise, experience has 
shown that there is no single "right" set of 

assumptions. There exists an almost infinite set of 

assumptions each more or less defensible. What is important 
is that the assumptions used in various submodels of a model 
are consistent. A model should not operate with one set of 
underlying assumptions in one submodel, while another 
submodel operates with a fundamentally different sat. 
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Because there is no emperical data concerning modern 
large scale combat, analysis is relied upon to produce 
insights into the effects of existing or proposed weapons, 
force structures, and strategies. Using theatre-level 
combat models, the analyst can examine proposed weapons and 
force structures and possible outcomes can be forcast. 
Studies in the DOD [6] have shown that when alternative 
force structures and weapons are examined using different 
models, it is difficult to determine whether differences in 
outcomes are due to differences in weapons and organizations 
or assumptions and models used. Confusion can be reduced 
if each alternative is examined in a consistent manner by 
using the same model and assumptions. Differences that 
occur can then be attributed to the basic structure of the 
force options. The results can then be analyzed and 
understood in light of the method of calculation {i.e. the 
model) used and its inherent assumptions. Greater insights 
into the effects of weapon and force structure alternatives 
on combat outcomes can be gained by repeating the experiment 
using an alternative model. 

When more than one model is used for the same force 
structure analysis often different results are obtained. 
These different results ere caused by the different inherent 
assumptions of each model. If these assumptions are known 
informed discussion can take place because differences can 
be resolved through evaluation of the assumptions. If the 
assumptions are valid and acceptable, then the results must 
be accepted as the logical consequence of the assumptions. 
Assumptions considered to be unacceptable by decision makers 
can be eliminated by modifying the model. What quickly 
becomes evident is that a model produces a result based on 
its inherent assumptions; an equally defeasible but 
different set of assumptions used in a model will produce 
another result, which may or may not be the same. 
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The DOD is quite aware of this aspect of modeling and 
has emphasized that there is more than one set of 
assumptions that can be used with a given model. It 

requires that all parties to the decision making process be 
aware of all assumptions leading to a result. To achieve 
this goal the DOD has required that all assumptions be 
explicit, reasonable, and consistent. This requires 
ferreting out hidden assumptions in models and insuring that 
the model indeed models what it claims to model. For 

interesting examples of the importance of assumptions see 
Ref. 6. 



C. MODEL EVALUATION 



Many solutions to the problem of model evaluation have 
been recommended; one concept suggests the use of full time 
dedicated independent reviewers as a way of improving the 
mechanics of quality control. [1] A reviewer provides a 
means through which the analyst’s model is scrutinized; the 
assumptions and methodology are checked for internal 
consistency, unwarranted inferences, and clarity of 
presentation so that a determination can be made whether or 
not the model is a plausible representation of the real 
world. If it is a large complex model, the methodology 
should be clear, even to the point of sample calculations to 
guide the reviewer and analyst through the algorithms. 
Equally important is that input data and assumptions be 
explicit. If unnecessary proliferation is to be avoided, 
existing models must be made understandable to potential 
users and evaluated in some manner. It should not be 
necessary to create a new model just because an existing 
model could not be transferred or understood due to 
inedquate documentation. 
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In his study, Gass [ 1 '4 ] proposes an elaborate approach 
to evaluation of complex models. Therein he highlights the 
need for model implementation and maintenance procedures as 
well as documentation of the model and the total modeling 
process. Suggested documentation of the process includes 
describing model objectives, assumptions, results, data 
sources, recommendations, etc. With such documentation the 
model can hopefully be evaluated and analysts can determine 
whether or not the model is valid for the problem at 
hand. However, Gass found that for most complex computer 
models, organizational exigencies and real-world pressures 
do not enable modelers to develop the necessary 
documentation. 



Gass has stressed the need to validate models at three 
distinct levels, technical, operational, and 

dynamic. Technical validity is an assessment of model, 
data, logical, mathematical and predictive 

validity. Operational validity is an assessment of errors 
and divergences found under technical validity and the 
robustness of the model (i.e. whether or not the model can 
produce bad answers for proper ranges of parameter values) . 
Dynamic validity is the method by which the model will be 
maintained during its life cycle so that it continues to be 
an acceptable representation of the real system. It 
includes the process through which the model structure is 
changed and validated. The ability to accomplish such 
validation will facilitate and enhance the use of models by 
anaylsts and decision makers other than those directly 
responsible for the development of the model. Fundamental 
to this process is detailed understanding of the model. 
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D 



SUMMARY 



If a model is to be of value it; must be accepted by the 
decision maker. It is incumbent upon the analyst to provide 
the means of acceptance. Sufficient documentation must be 
provided by the model designer to enable the analyst to 
understand its methodology and structure. Key to 
credibility is objective evaluation. Documentation must 
provide insights into the assumptions and functioning of a 
model, so that it can be evaluated. Understanding and 
evaluation are complicated by complexity. Complexity and 
insufficient documentation can cause analysts to design new 
models rather than use an existing model. The existence of 
inadequately documented models describing the same reality 
is the basic cause of the criticism and lack of credibility 
of combat models. The next chapter will discuss complexity 
and how and why it contributes to unnecessary proliferation. 
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III. PROLIFERATION,*. COMPLEXITY^ AND TRANSPARENCY 



Unnecessay proliferation is the cause of some of the 
criticisms of combat modeling. Unnecessary proliferation 
means that a new model is created because inadequate 

documentation precludes the use of an existing model which 
otherwise would be adequate for the task at 

hand. Inadequate documentation either prevents 

understanding of the methodology and structure of the model 
or prevents the cost-effective transfer and conversion of 
the model from one agency and computing system to 

another. The latter condition is the one encountered by the 
author during the conversion of ATLAS. Better documentation 
would have significantly reduced the resources expended on 
the project. Futhermore, in a non-academic environment the 
demands of proceeding with an impending study would have 
encouraged analysts to develop a new model cather than 

struggle with poor or non-existing documentation. 

Development of a new model can cause analysts to remodel 
a combat process using techniques and methodology that 
already exists; consequently funds are expended without 
advancing modeling. Irrespective of any advancements in 
techniques or improvements in methodology developed in 
remodeling an existing process, a major portion of the 
effort is devoted to cedoing the basics. Time and money 
spent in redoing the basics are lost as far as improving 
models and modeling is concerned. In the DOD (especially 
the Department of the Army) there has been and continues to 
be a shortage of trained analysts. An obvious approach to 
alleviate this problem would be to eliminate any projects to 
develop models that would be redundant and share existing 



21 



models. However, the ability to share a model is greatly 
influenced by its complexity. 



When examining the practicality and feasibility of model 
sharing most emphasis has been placed on 

documentation. Huber [8] lists poor documentation and high 
personnel turnover as a prime reasons why models are not 
exchanged. The criterion used to determine whether or not a 
model is transferable is documentation sufficient to allow 
the recipient organization to be able to run the model 
without an inordinate amount of decoding and 
deciphering. Without adeguate documentation the model is a 
puzzle to the recipient. Shubiclc and Brewer [10] found that 
few models exist in a sufficiently documented 
would satisfy commercial firm standards 
distribution to their clientle. 



form that 
prior to 



A. PROLIFERATION AND HOW TO REDUCE IT 



An area examined by the Comptroller General in 1974, 
[15] was sharing of computerized models. Models developed 
for specific purposes by one agency can often be used by 
other agencies for simiLar purposes. Applicability of 

models to new situations depends on their accuracy, purpose, 
validity, availability of sufficient documentation, and 
capability of the using analyst. The 1974 study surveyed 
242 models that had a combined cost of over thirteen million 
dollars. An attempt was then made to obtain documentation 
for about one hundred randomly selected models deemed to 
have use at more than the originating agency. Information 
explaining the purpose, mathematical formulation, and 
operating instructions ware not available for approximately 
one-third of the models. The survey identified the 

primary complaints of model users (programmers and analysts) 



22 



as: operating instructions not available or not clear, 
hence, compounding the already difficult problem of 
preparing a model for use on a different computer; 
mathematics of the model not clearly explained, hence, 
restricting the understanding of the logic, capabilities and 
limitations of the model; sample inputs and outputs 

nonmatching or nonexistent or do not correspond with the 
sample data in documentation, hence, verification and 
validity of the model difficult to determine; flow chart 
explaining logic not provided or not current, hence, complex 
subroutines not easily understood. The investigation 

determined that benefits can be obtained by sharing computer 
models, however, before models can oe shared adequate 
documentation must be prepared. Such documentation enabled 
the acquiring agency to determine whether a model met its 
needs and was the primary factor to successful conversion 
and operation of the model on a different computer. The 
potential for a cost-effective transfer is severely limited 
in the absence of adeguate documentation. When such 

documentation is available to potential users of an existing 
model great savings are realized. An example was the 
transfer of a complex communications traffic analysis model 
from an Air Force agency on the West Coast to the Systems 
Development Command at Hanscom Air Field in Bedford, 
Massachusetts. [15] 

Joint usage of existing models not only increases the 
availability of trained individuals to do further research 
and analysis, but it reduces the opportunity that different 
factions of the same organization are working at cross 
purposes. Conflicting concepts and proposals are necessary 
to vitalize an organization and make it viable, but 
developing a conflicting position that could be resolved 
prior to the expenditure of great sums of money and 
analytical talent is a waste. The sharing of a model or 
models between two conflicting agencies allows each to 
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understand the underlying basis for their respective 

positions concerning an analysis of a decision making 
situation. While sharing may not resolve the conflict it 
certainly will preclude the expenditure of funds in the 
independent acquisition of information that is already 
available through the medium of an existing model. Sharing 
has eliminated retracing steps already taken and dead ends 
already discovered when applied in other fields , applied to 
combat modeling sharing will provide these minimal returns 
and has the prospective of providing even greater 
returns. If the basics are not reprocessed then more time 
and money is available for modifications to enhance the 
capability of an existing model, correct known deficiencies, 
or identify suspected deficiencies. Sharing has promise to 
improve the economies of modeling. [16] 

Before sharing can be achieved certain basic conditiions 
must prevail. Five necessary conditions [17] to model 

sharing have been found. They are; 

(1) a computer able to use the program with minimal 
modif ica tion ; 

(2) an adequate facility to run the model; 

(3) adequate documentation of the original model; 

(4) sufficient analysts with technical competence; 

(5) formalized arrangements for sharing costs and 
responsibility for costs and coordination. 
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B 



THE EFFECTS OF COMPLEXITY OH TRANSPARENCY 



Complexity is a dichotomous issue in itself. Gass 
perceives an increasing use of more complex models [14], 
while Hardison, the Deputy Under Secretary of the Army for 
Operations Research at the 15tn Annual US Army Operation 
Research Symposium called for less complexity. 
Disatisfaction of both senior military and civilian decision 
makers with complex models and studies was emphasized by 
Hardison. [ 18] These decision makers are convinced that 
Army models and the studies they support are too complex, 
elaborate the obvious, belabor needless details and overlook 
key issues. Timeliness is also affected by 

complexity. Failure to delimit results in failure to meet 
schedules which causes something to be sacrificed; often, in 
the case of models that something is adequate 
documentation. 



A corollary of the complexity issue is that of 
transparency. If all the interactions of a model are to be 
understood by both the analyst and the decision maker it 
must be structured and programmed so that its methodology is 
easily understood. A model that fulfills this requirement 
is said to be transparent. At the 35th Session of MORS a 
leading cause of the general disenchantment with 

theatre- level models in recent years was attributed to a 
lack of transparency in most models. The proposed 

resolution to this problem was to include in the model only 
those interactions and factors that can be shown to 

influence the outcome. This in combination with 

mathematical formulation that is as simple as possible 
should produce the desired transparency. [19] Yet, at this 
same MORS session A.H. Cordesman OASD (Intelligence) in 
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remarked 



that 



discussing theatre- level models remarked that models 
currently being developed go into unnecessary levels of 
detail in ways which seriously limit their value. This is 
partially caused by intermediate managers and decision 
makers requesting particular attributes be modeled. At 
times modeling efforts deviate from the maxim that only 
"essential" variables be modeled. As Morris [2] suggests, 
the purpose of a model is to include only those variables 
that characterize the process being modeled. At present 
diametrically opposed forces exist; while expounding the 
need to maintain models at a simple transparent level 
current modeling efforts go into details that detract from 
transparency. 



A simple solution to the transparency-complexity problem 
may not be easily obtained. In spite of the professional 
rhetoric to the contrary, Gass [14] has found an increase in 
the use of complex models at all levels of government and 
industry. He attributes this to better trained analysts and 
the development and refinement of methodologies. Although 
simple models with readily understood assumptions, 

relationships and structure are preferred, Gass contends 
that decision making problem environments representative of 
the Federal Government sphere cannot be realistically or 
logically contained by simple models. Furthermore, senior 
decision makers generally do not possess a detailed 
understanding and appreciation of the methodologies employed 
in the various models employed to assist them in the 

decision making process. What is needed is a method 

through which the use and interpretation of the outputs of 
models by senior decision makers is facilitated. A model is 
usable only if it is understood and plausible to analysts 
and decision makers. They (particularly the decision maker) 
must be given the opportunity to explore the use of the 
model, become familiar with its predictions, and examine the 
relationships and assumptions implied by the model. In 
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C. SUMMAS? 



To reduce unnecessary proliferation and reduce costs the 
Comptroller General has recommended model sharing when 
feasible. Sharing a model requires: 

■ ability to use it with minimal conversion; 

■ adequate facilities to run the model; 

■ sufficient competent analysts and programmers; 

■ adequate documentaton; 

■ formalized arrangements for cost sharing and 
coordination. 



Findings indicated that the great complexity of 
theatre- level models coupled with rapid turnover of 
personnel has resulted in models being used as "black boxes" 
with neither the computer technicians that run the model nor 
the analysts knowing explicitly what or how the model 
operated on the input data to provide the final results or 
output. Hence, the analyst was unable to adequately explain 
the results to the decision maker; with each occurence the 
credibility of modeling diminished. Concurrently models 
had proliferated to such a degree that the turnover of 
personnel exacerbated an already critical personnel shortage 
situation. Amelioration requires reduction of the number of 
models in use and detailed justification before developing a 
new model. [1] Reduction or the minimization of the growth 
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of the model inventory implicitly requires that models be 
easily and quickly moveable from one usinq agency to 
another, i. e. posses transferability. Key to the resolution 
of the problems of complexity and transparency is 
document ation. 

« 

Complexity may be an unavoidable recourse of combat 
modeling because of the demands of managers and decision 
makers. Unless complex models are sufficiently documented 
to make them readily understandable and usable, analysts 
will create a new model. Rather than creating new models 
an atmosphere conducive to the sharing of models should be 
incouraged. Requirements to transfer a model are discussed 
in the next chapter. 
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IV. REQU IRE MENTS TO TRANSFER A MODEL 



The Review of Selected Army Models report [ 1 ] proposes 
that the number of models retained by the Dept, of the Army 
be reduced and that existing models be used where possible 
before a new model is commissioned. To properly evaluate an 
existing model a method for analyzing and verifying a 
candidate or candidates from existing model resources must 
be established. The more complex a model is the more 
difficult is this analysis and verification. [14] 

Use of an existing model requires understanding of 
potential problems due to model design as well those 
problems expected to be encountered during transfer and 
execution. Design problems can include lack of adequate 
sub-models, failure to consider key variables, inaccuracies 
or lack of validation, computational difficulties, and 
inconsistent hidden assumptions. Problems during exection 
can be those of irrelevance, inadequacy of output, 
inappropriateness of assumptions, lack of connection to 
other models and results, statistical and extrapolation 
difficulties . 

A. METHODOLOGY FOR ANALYSIS OF EXISTING MODELS 

3efore an existing model can be used it must be analyzed 
by the potential user. Examination of the literature for a 
methodology for conduct of an analysis generally produces a 
concensus. Such methodologies center on five general areas. 
[ 20 ] These are: 
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(1) inputs and outputs, i.e. the global structure of the 
model ; 

(2) the basic causal relationships assumed between 
variables; i.e. the local structure of the model; 

(3) the detailed logic of the model; 

(4) the numerical values of the data, and 

(5) the time and resources required to exercise the 
model . 

Additionally, an analysis should consider any 
experimental studies that allow comparison of the model 
predictions with the real world, other models or with the 
intuitive beliefs of the decision maker that will ultimately 
be presented the outputs and their interpretation. Such 
previous studies are useful in evaluation of a model for 
application to new problems or situations. Unfortunately, 
so far as combat models are concerned, comparisons with real 
world results are extremely limited. When such data is 
available (e.g. WWII and Korea) it is sparse, subject to 
conflicting interpretation, and of questionable accuracy. 
[ 21 ] 



A detailed examination of the glooal structure of a 
model can forthrightly answer the basic question of such an 
analysis. Is this modal capable of examining the problem 
at hand? The potential user is interested with whether the 
outputs measure the desired quantities and or qualities, and 
whether the desired inputs can be entered into the model in 
the form in which they are available. Using an existing 
model is not cost-effective if available input data must 
undergo a costly and time consuming conversion. There must 
be provisions in the modal structure to allow changes in the 
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input data to reflect changes in the combat process under 
examination . 

The local structure is examined to expose important 
causal influences which may have been omitted for ease of 
computational effort or other reasons. This step is closely 
related to the abstraction process in the design of the 
model. It is most dependent on the detail and completeness 
of available documentation. It is critical because the 
importance of a causal connection can be quite subtle but 
very pervasive. 

Examination of the detailed logic of a model identifies 
the hypothesis upon which the model is based. It reveals 
extent to which it is based on historical and experimental 
field data. It is another indicator of the appropriateness 
of the model to solve the problem at hand. This step also 
reveals the presence of inconsistent assumptions or 
inappropriate assumptions. 

Finally/ analysis o£ the time and resources required by 
the model provides the means through which the costs 
associated with the use of the model can be determined. To 
make rapid and accurate estimates the description of the 
model must provide explicit information of time requirements 
to gather and prepare required inputs as well as the time to 
execute the model given the inputs. 

If models are to be truly transferable between agencies 
the above analysis must be comleted prior to any attempt to 
transfer a model. Accurate and complete documentation must 
be available if this is to be conducted. Such 

documentation will be available only if it is prepared 
concurrently with the development of a new model and in 
concert with any modifications made during the life of the 
model. 
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B 



POTENTIAL PROBLEMS IN EXISTING MODELS 



To adequately analyze an existing model the analyst must 
be familiar with potential problem areas. Farrell [20] has 
suggested that the analyst consider: 

(1) the adequacy of submodels, i.e. do they model what 
they purport to model; 

(2) whether key variables were overlooked during the 
process of abstraction; 

(3) the possibility of inherent inaccuracies; 

(4) possible computational difficulties; 

(5) whether the model has inconsistent and inapproriate 
assumptions ; 

(6) possible inadequacy of output. 

Among the problems of design can be found cases where 
the general model structure is adequate but reasonable 
submodels are not available. This is particularly true of 
sub-models involving simulated tactical or strategic 
decision making. Farrell [20] has indicated that most 
diagrams and flow charts of such sub-models do not reveal 
the sub- model inadequacy. 

In designing a model the first and most difficult step 
is insightful abstraction by the modeler or modelers. To 
abstract the real world into a representative model, key 
variables, their basic causal relations and interpretation 



32 



must be incorporated into the model. This step is very 
difficult to verbalize since any description of the process 
to be modeled will necessarily incorporate the results of 
abstraction in the elements of the description. This step 
is imaginative, creative, and complex; it is imperative that 
it be documented at the time of abstraction. Once a 

process has been modeled and time passes the explicit steps 
and reasoning thru which the abstraction was made cannot be 
adequately described by the modelers. Failure to consider 
key variables occurs at this almost invisible step of 
abstracting a process of the real-world into a model or 
sub-model. 

In rhe review of an existing model it is these almost 
invisible steps of abstraction that must be thoroughly 
examined to insure all key variables are included in the 
model. Key variables can also be excluded because of lack 
of adequate sub-models or computational problems their 
inclusion or manipulation would precipetate. The reviewer 
or user of an existing model must carefully search for key 
variables that have not been modeled or are simply thruput 
in the model. 

Another pitfall hidden in the design of existing models 
is inaccuracy or lack of validation. Often because of 
computational difficulties known experimental results have 
not been included in a particular model. This occurs most 
often as a result of exigencies on the modeler or modelers 
at the time the model is created. Exclusion of such 

experimental data results in inaccuracies. Lack of 
experimental support for portions of military models is 
another common cause of inacuracies which cannot always be 
avoided. Likewise, inadequate debugging of the model can 
be a reality. This is not serious as long as these 

inaccuracies are known. Unfortunately, these facts can 

escape a potential user if a careful review of the model 
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design and its use is not made. 

A thorough review of modal documentation can reveal 
computational difficulties if the documentation is 
adequate. These difficulties usually involve either 
algorithmic imprecision or excessive date storage 
requirements. These limitations should be revealed by the 
documentation to the potential user so that proper use of 
the model can be made, unexpected or excessive costs are not 
incurred or another model can be selected. 

The bane of analyst using an existing model is 
inconsistant hidden assumptions. Such assumptions included 
as part of the overall model can be at odds with those of 
sub-models, the data base, and data generating routines. 

The most pervasive error awaiting the unwary user of an 
existing model is the inappropriate ne ss of inherent 
assumptions. This potential error is the most difficult 
to detect when using a pre-designed combat model, since the 
user is generally not well versed in the process through 
which the abstraction of the real world was made. During 
the abstraction fundamental assumptions concerning the 
nature of the combat process as well as assumptions for 
computational reasons were made. Only through careful study 
of the line of reasoning followed at the creation can a 
would-be user become familiar, with these assumptions. Care 
must be taken so that not only the explicit but also the 
implicit assumptions are understood and their effect on the 
combat process being modeled is 

comprehended. Unfortunately, the reasoning and logic 
followed in creation of a model are not included in current 
documentation. 
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If there are potential problems awaiting the unwary 
analyst in the design of an existing model, these are just 
the forebodings of greater problems associated with the 
actual exercise of the model. Foremost of these is 

irrelevance, that is caused by either attempting to use a 
model on a problem for which it was not designed or a 
failure to understand the problem thoroughly enough to 
select an appropriate model. 

Concomitant with irrelevance is inadequacy of 
output. This results when useful information is inherent in 
the model itself but actual calculation and or display does 
not exist. Causes of such conditions are selection of an 
improper model or laclc of full understanding of the 
intricacies of the model. Proper and adequate 

documentation and careful perusal will ameliorate the 
situation . 

In complex simulations, both situations and key 
parameters will be varied and the results examined. The 
number of unique situations examined is limited by resource 
and time constraints, because of this the potential user 
must be sure that required outputs are readily available. 

Adequate descriptions of the output of any random 
process are difficult to achieve under the best 

conditions. Under time constraints descriptions of the type 
of output provided by a model as well as how the outputs are 
collected is critical. Without it, the user cannot 

correctly interpret the results obtained. When available 
time and resources preclude simulating all situations the 
problem of extrapolating or interpolating between the 

particular situations modeled arises. There is no adequate 
general method for surmounting this problem; the problem is 
less severe the less complex is the model being used. Any 
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statistical difficulties encountered only compound the 
problem. Surmounting this problem is a function of the 
ingenuity of the analyst and the detail of the documentation 
available to him. In the case of theatre- level combat 
models obviously, neither all situations can be simulated 
nor are there adequate historical or experimental data with 
which to compare the results obtained. A thorough 

accurate and consistent interpretation of the outputs of 
such a model is highly dependent on the analysts intimate 
familiarity with the mathematical structure within the model 
as well as interactions between the various input and output 
data. Such intimacy is obtainable only through available 
documentation if the analyst uses an existing model designed 
by someone else. 

C. SUMMARY 
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■ inputs and outputs; 

■ basic causal relationships; 

■ model logic and structure; 

■ available data and required data; 

■ time and resources required. 



Many potential problems are contained in an existing 
model, among these are: 

■ submodel inadequacy; 

■ key variables excluded; 

■ inherent inaccuracies; 

■ computational difficulties; 
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m inconsistent assumptions; 

■ inappropriate as-sump tions; 

■ inadequate output. 

The ability and to what degree model analysis and 
consideration of potential problems can be accomplished is 
determined to a great degree by available 

documentation. Adequacy of documentation is considered in 
the next chapter. 
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STATUS AND ADEQUACY OF HORRENT DOCUMENTATION 



V . 



A. DEFINITIONS AND BACKGROUND 



Program documentation is a collection of information to 
explain the design, development, and maintenance of the 
program as well as purposes, methods, logic, relationships, 
capabilities and limitations. [5] Except for the simplest 
programs it is difficult, if not impossible, for someone 
other than the originator to determine what is supposed to 
be accomplished by just reading the program code. 

Documentation is necessary for: planning, programming, 

managing, operating and evaluating models. It is absolutely 
essential for: quick and effective changes; use of the 

model by programmers and analysts other than the 
originators; understanding of what is being done; 

interagency program sharing; verification of proper model 
operation. Through adequate documentation secondary users 
gain an understanding of a model and thus the model and its 
outputs are rewarded a level of credibility. It is vital 
if secondary users are to be able to run the model and make 
necessary modifications of the program. This restrains 
proliferation and duplication which can result in major 
savings; besides it tempers an already complex 

environment. Unfortunately, current documentation practices 
are such that the documentation to facilitate the use of 
existing models generally does not exist. 
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B. CURRENT DOCUMENTATION PRACTICES 



Although documentation is an aspect of computer 
programming that was recognized from the inception of 
automatic data processing as being critical to successful 
programming, it has been and still is a major problem 
area. Unfortunately, for various reasons including time and 
fiscal constraints as well as lack of definitive guidance 
concerning requirements and standards, the bulk of the work 
effort concerning documentation has consisted of unfulfilled 
requirements. Notwithstanding the fact that programming 
has existed since the inception of ENIAC in 1944, the lack 
of adequate documentation received major attention in 
studies concerning models and simulations as well as 
investigations by the Comptroller General of the Army in tae 
late sixties and early seventies. [10,15] 

Irrespective of the aforementioned studies the problem 
of inadequate documentation persisted and was the subject of 
another investigation oy the Comptroller General in 
1974. [15] Over seventy federal installations in the 
continental United States, Europe, and Asia were 
surveyed. These included selected DOD Agencies as well as 
those of each of the armed forces. 

The study cited several problem areas attributable to 
inadequate documentation. Increased cost of operations was 
high on the list. Because of inadequate documentation use 
of operating programs is hindered since current operators do 
not fully understand how and what is being done in a 
program. Therefore, when unexpected outputs are obtained 
it is difficult if not impossible to determine their 
validity. Equally perplexing is inadequate documentation of 
subsequent modifications incorporated into the model. In 
many cases without adequate documentation it was impossible 
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for a new analyst an} or programmer to use or modify an 
existing program. Ultimately such models have to be 
completely rewritten or the time required to make them 
usable was greatly in excess of the time required when 
adequate documentation was available. In a cited example 
inadequate documentation caused an agency to spend over a 
year to determine how a particular complex model 
operated. Another example indicated the difference of six 
man-months of work incurred because of the lack of 
sufficient documentation. Although the Comptroller General 
found it difficult to determine the aggregate cost of 
increased operating expenses due to inadequate 
documentation, he did indicate it was high because of the 
number of cases uncovered. 



Lack of sufficient documentation was the major cause of 
the problems encountered by the author during the conversion 
of the ATLAS model. tfhen machine differences required 
changing the program code there was minimal guidance as to 
which sections of the code corresponded to " particular 
functions described in the user's manual. *22] Also, 
details of the mathematical structure of ATLAS are not 
contained in the manual. For details of the model 
structure references are made to a models manual [23] for 
the predecessor of ATLAS. The extent to which the structure 
of this model has been incorporated into ATLAS is not 
explicitly stated. The situation is further complicated by 
the fact that pertinent assumptions basic to the model 
formulae are not in the models manual; they are in the 
user's manual listed in a haphazard manner. Since the 
models discussed were originally designed for the 
predecessor of ATLAS there is no assurance that changes to 
the model formulae were not made during the evolution of 
ATLAS. This suspicion is enhanced by the fact that in 1973, 
five years after the design of ATLAS, a significant 
programming error was found. [24] 
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There are a myriad of reasons why model documentation is 
inadequate. One of the primary causes is that documentation 
guidelines and policies are developed by individual Federal 
departments and agencies. At the highest levels the 
guidance is necessarily general, as it moves down the 
organization further more explicit implementing directives 
are provided culminating in directives issued by the 
developing agency. Hence, some documentation is brief and 
simplistic; other documentation is detailed, voluminous and 
complex; neither may prove to be adequate. Adequacy is 
determined by the ability of other than the originators to 
use and understand the model. The Comptroller General found 
that even when guidelines and standards were prescribed 
managers of modeling projects failed to insure 
compliance. The type and content of documentation is often 
decided by computer technicians or ADP operators. [15] 
Shubik [17] has cited this practice as unprofessional since 
ambitious programmers have been known to change coding in 
the pursuit of computing efficiency without making note of 
the fact. 

An examination of 264 model documentation packages at 10 
California installations revealed none fully complied with 
the agency standards and most were incomplete, inconsistant , 
and inadequate. [15] In most cases programmers determined 
what documentation to prepare based on their own 

judgement. Managers responsive for developing models 
indicated that the reason standards were not adhered to was 
because of time contraints. Completion dates frequently 
were given precedence over preparation of adequate 
documentation. The desire to complete a model and get it 
operational by a given date overrode the need for 

documentation. 
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An aspect of managerial responsibilities concerning 
documentation is that it be kept current. Even if a model 
is initially adequately documented it will eventually be 
modified and its documentation must be likewise 
updated. This is a major problem because it requires 
diligence to update documentation. Given time and resource 
constraints and the exigencies of the decision making 
process it is a task that can easily be put off since the 
model will work without such documentation. The problem 

comes later when the personnel that modified the program 
become involved with other demanding problems, leave or are 
transferred. Later the documentation is difficult to 
prepare because the reasons why or how the modification was 
made become unclear or those that knew what was done are no 
longer available. 

There are numerous comment cards in the ATLAS program 
that indicate changes were made. However, there is no 
formal documentation, with one exception, to indicate what 
or how these changes were made. Informal documentation 
provided was minimal and superficial and did not address all 
the indicated changes. The one exception was a formal 
document [25] prepared in 1974, which discussed improvements 
in the treatment of barriers and personnel replacements. A 
global variable and subroutine listing were provided but 
these were not up to date; had they been current the 
conversion would have been facilitated. 

Poor or nonexisting documentation persists irrespective 
of the efforts of the Comptroller General and the Department 
of Defense. Inspite of the identification of this problem 
early in this decade [1,10,15] its presence continues to be 
a problem at the close of the decade. [3] The continuing 
lack of detailed documentation that enables an analyst to 
understand what goes on inside a model is cited by Shupack 
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[26] in 1979, as a limiting factor in the use of 
theatre- level combat models. Although there was a 

noticeable improvement, he found the level of documentation 
of IDAGAM not sufficient to insure the easy and proper use 
of the model without supplementing the available 
documentation. Without adequate documentation the analyst 
is apt to make erroneous conclusions regarding the processes 
occurring within a model. 

Although documentation problems persist genuine attempts 
to resolve the problems have been made. Current combat 
modeling efforts at the Naval Postgraduate School are not 
only using languages especially designed for simulation but 
the documentation methods employed enhance the transparency 
of the model. For examples see Ref. 27, 28, 29. Other 

agencies have also made inroads toward improving the 
adequacy of documentation. See Ref. 30, 31, 

32. Irrespective of these improvements the author believes 
a vital aspect of documentation is being overlooked. This 
aspect and appropriate recommendations are set forth in the 
next chapter. 



C. SUMMARY 



Studies as well as intuition reveal that to understand 
the workings of a complex model an analysts requires a 
detailed explanation of the calculations and data 
manipulation performed by a simulation. Implicit and 
explicit assumptions and inputs must be known if an analyst 
is not going to use a model as a black box. A detailed 

knowledge of the variables, subprograms and their 
relationships must be acquired if a model is to be 
transferred from one using agency to another. Rarely will 
both organizations posses the same brand of data processing 



43 



equipment, more likely than not there will be great 
dissimilarities. Conversions from one machine to another 



requires that changes be made to the model. To 
model the programmer and or analyst must know the 
change will have not on only the particular line 
is changing but throughout the program. Analysts 
only know how a change will affect the 
minipula tions and computations of the various part 
model but how it will interact with the implict an 
assumptions of the model. To gain this 
comprehension of a model designed by someone 
analyst and programmer of the gaining organize 
consult the documentation provided with the 
Without such documentation analysts have chosen to 
a new model. Unfortunately, the general concen 
investigations was that documentation was of dubio 
and generally inadequate. [1,10,15] 
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PROPOSED HIERARCHY OF DOCUMENTATION 



The discussion of who the user is, the role of the 
officer analyst, and the documentatio n concept that follows 
is based on seven years experience in operations research 
related assignments, discussions with analysts, programmers 
and students, and problems encountered during conversion of 
the ATLAS model. The author has drawn upon the above 

sources and selected literature [4, 15, 17, 33, 34, 35] to 

propose a new concept in model documentation. This new 
concept is intended to refine and supplement current 

documentation methods. Implementation of the proposed 
concept will hopefully fill a void that currently exists and 
will greatly improve the transparency and transferability of 
combat model. 

A. THE USER AND THE ANALYST 
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The analyst is the manipulator of the model. He is 
responsible for using the model to support the study 
objectives. This means he is responsible for the collection 
and insertion of input data and interpretation of the output 
data. If the model is small, he may do the coding; or if it 
is complex, a programmer may assist him in the coding 
process. Any interpretation of how the model represents 
reality is the analyst's responsibility. To fulfill this 
responsibility he must be familiar with the conceptual basis 
of the model, its underlying assumptions, how the model was 
transformed into the program code, and how the code operates 
to present the process modeled. The analyst must know if 
any changes to the conceptual foundation of the model 
occurred in the process of programming (coding) . 

There is a reluctance on the part of many military 
analysts to gain an intimate understanding of a complex 
combat modal. Exingencies of the organization are some of 
the prime reasons. In the daily demands to manipulate a 
model and to produce results there is insufficient 
allocation of time to study the inner workings of the 
model. Reliance is placed almost exclusively on the user's 
manual to explain the results produced. 

Some analysts (this is particularly true of the officer 
analyst) refuse to gain detailed understanding of the inner 
workings of a model because they do not perceive that to be 
their function. Their perception is that they are the user 
and do not require that level of detailed knowledge. Others 
hesitate to learn or become well versed in the functions of 
a particular model because they fear such expertise will 
label them and hence limit the scope of their future 
assignments. In discussions with officer analyst students 
and practicing officer analysts, the author found these 
attitudes to be quite pervasive. Many considered detailed 
knowledge of how a model operated to be in the realm of the 
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programmer's responsibility. Little or no consideration was 
given to the fact that most programmers have little or no 
military background and cannot possibly relate the 
machinations of the program code to the combat process. The 
programmer views proper operation in terms of proper 
execution of programmed instructions. Whether or not 

instructions properly depict the realities of the combat 
process are beyond their scope of interest. They have 
neither knowledge, inclination or experience to evaluate the 
fidelity of the code. 

Attitudinal attributes of military analysts can be 
understood in light of their rapid turnover in 
assignments. Rapidity of reassignment discourages the 
desire • to gain detailed knowledge of any particular 
model. Most likely they will be reassigned shortly, to 
other type duties. If subsequent assignments deal with 
combat models, most likely, it will be a different 

model. Any time expended in the study of details of the 
model associated with the current assignment is considered 
of minimal value. When an effort is made to understand a 
model it is often frustrated by inadequate documentation. 

Probably the most frequent attitude seen in officer 
analysts is the one associated with the military psyche, 
that of being a generalist. This attitude is the result 
of the total experience of the profession; it has been the 
way to reach success in the past, and many believe that it 
still is the way to success. Although there has been 
considerable effort to change this mind set, the example of 
the last few centuries is difficult to overcome. Compared 
to the existence of the military profession, the experience 
of the military analyst is of recent vintage. Only 
favorable experience will provide the impetus for change. 
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The transfer of models between agencies will contribute 
favorably toward convincing analysts to gain detailed 
knowledge of combat models. If they know that the effort 
expended now can be used at a later time in some future 
assignment, the effort will be considered worthwhile. But 
adequate documentation is a prerequisite of such 
transferability. Hence, there is an interdependency 
betweeen documentation and convincing military analysts to 
gain a detailed understanding of combat models. If they can 
see value in the expenditure of tae time and effort, they 
will be willing to make the commitment. 



Shubik £16] has found that the mathematical modeler and 
the person who understands the reality the model attempts to 
abstract are not necessarily the same person. The combat 
process is best understood by senior military decision 
makers who are generally unable to translate it into the 
appropriate abstraction and who generally desire greater 
detail than is necessary. A good model is one that is able 
to abstract and describe only that which is relevant to the 
problem under investigation. On the other hand the 
mathematical modeler is frequently an individual who 
generally lacks the experience and an appreciation of the 
nuances of combat which can lead to the development of an 
ill-structured model. The military officer analyst 

represents a step toward providing a modeler or model 
operator who not only understands the mathematical aspects 
but the military factors as well. If this capability is to 
be maximized when dealing with existing models, there must 
be documentation which allows the analyst to link the 
conceptual model formulation to the executable 
program. Then he can understand the conceptual basis of the 
model, insure the program fulfills the concept, and act as a 
source of information for the decision maker. 
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a. ORIGIN OF THE DOCU MENTATION SHORTFALL 



The initial theatre- level combat models used in the 
United States were designed and operated by contract 
organizations (e.g. Research Analysis Corp.). These models 
had a manual referred to as a user's manual which provided 
an overview of the model, input requirements, and a 
description of the output. For an example see Ref. 22. 
Documentation within the manual was superficial, at most it 
provided the proverbial big picture. Sufficient 
information was provided to determine the general 
capabilities of the model, possible suitability for a 
particular study and a general indication of expected 
outputs. It provided little if any information on the 
insights that could be attained or subtleties of the 
model. The user for whom this "user's manual" was designed 
was a project manager or some level of decision maker. The 
details could be filled in by the persons who would operate 
the model, this most likely was the designer of the 
model. Because of budgetary reasons and doubts that these 
model operaters understood the nuances of military 
operations, the decision was made to bring these models 
directly under Dept, of the Army control. 

When these models passed "to Army control all 
documentation passed with them. Unfortunately, in most 
cases the documentation was minimal consisting primarily of 
program listings, operator instructions, global variable and 
subroutine listings, flowcharts, user's manual and possibly 
some limited discussion of the formulation of the model. 
These models were placed under control of the agency that 
would use them in support of force planning and strategy 
studies. The designated users that inherited these models 
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were the military analysts (officers and federal employees) 
assigned to the organization that received the models. The 
analysts then took the model and the user’s manual and 
proceeded to exercise the model in support of on-going 
projects. 

Use of the user's maaual by the analyst was mandated by 
the fact that input requirements and preparation were 
contained in it. However, when the models were initially 
transferred from the designing corporation to the Army no 
supplemental documentation was prepared or 

provided. Analysts were using a manual that provided only a 
superficial examination of the model. They performed 

analyses using a manual originally designed for a study 
director or decision maker. These "user's manuals" did not 
contain the detailed information of the model that is 
necessary if the analyst is to know and understand the 
machinations through which the input data is exposed to 
produce a given set of outputs. 

Evenually conflicting results were obtained from models 
supposedly examining the same situation. When called upon 
to resolve or explain these discrepancies the analysts were 
unable to do so. To explain the conflicts a detailed 
understanding of the conceptual basis of the model as well 
as detailed information of the translation of the conceptual 
model into program code was needed. Only with this 
information could the analyst explain why two models 
examining the same combat process under the same conditions 
produced different results. 

Analysts then discovered that the documentation provided 
with the model did not provide this insightful 

information. To answer the question the model designer had 
to be contacted. At times this was impossible because the 
actual model the designer no longer was with the firm that 
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nor 



was 



developed the model, nor was there available any 

supplemental information on the inner workings of the model. 
Instances in which the designer could be located usually 
proved just as unrewarding. In most cases the designer had 
no supplemental formal documentation. When he acted as the 

operator of the model on behalf of the Army he could answer 
such questions based on his design knowledge and daily 
contract with the modal. Since there had been no formal 
requirement for such supplemental information, it never was 
prepared. Once the model passed under operational control 
of the Army, the designer was precluded from daily 
contact. Over time the designer's intimate familiarity with 
the model waned, especially the intracacies of the 
translation from basic concept to operating program 
code. If the designer had not done the actually coding, 
similar results were usually experienced when trying to 
locate the original programmer or obtain supplemental 
information. What the program code was actually doing was 
not readily apparent and documentation or personal knowledge 
of its operation were unavailable. The situation is best 
expressed by a quote from Robert Frost. When once asked by 
a critic what he meant by a particular phrase in one of his 
works, he replied, "When I wrote it Sod and I knew what it 
meant, now only God knows". This same situation will also 
prevail with combat models unless adequate documentation is 
created concurrent with the design and development of the 
model. 



C. 



COMMUNICATION 



AND LEVELS OF PARTICIPATION 



The prime purpos 
communication of why a 
condensed into a form 
predict a future state 



e of documentation is communication: 
nd how realities are abstracted and 
suitable for exercise on a computer to 
of some combat process. Generally 
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the level of professional communication between decision 
makers, analysts, and modelers has been low as attested by 
the credibility problem previously discussed. This coupled 
with the turnover of military decision makers and analysts 
has not enhanced comprehension of and control over 
large-scale combat models. Very often the information about 
context and limitations of these models has not been fully 
understood by the military analyst and therefore not clearly 
presented to the decision maker. Documentation that can 
provide such enlightenment to the analyst will be a 
significant: step toward providing the required insights into 
combat models needed by decision makers. Through adequate 
documentation the user and analyst gain understanding, and 
with understanding can come acceptance and the decision to 
use the particular model. 



In combat modeling there are three levels of 
participation with necessary intercommunication. At the 
highest level is the decision maker who uses the model as an 
aid to gain insights and in conjunction with judgement 
arrive at a decision concerning force structure or 

strategy. The intermediate level is occupied by the 
analyst, who either designs a model or uses an existing 
model; prepares inputs; exercises the model and interprets 
the outputs. If the model is new to the decision maker he 
also aids in the model selection process by recommending and 
explaining the inherent attributes of particular models in 
pursuit of a projected study. At the lowest level is the 
programmer, who writes the necessary code during model 
construction, explains the limitations of execution of a 
model on a particular computer, insures the model is 
processed as the program intended it and to code necessary 
modifications to a model. Each level has unique 

requirements and responsibilities and therefore the 

documentation required by each is unique. 
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The decision maker uses his documentation to gain an 
overall appreciation of a model and sufficient understanding 
to question the analyst on model details necessary to make a 
decision to employ the model and use its outputs. It 
provides a means of conducting discussions that can reveal 
what insights can be gained from a particular model. The 
decision maker uses his documentation as a link between the 
analyst and himself. 

The analyst occupies the pivotal position between the 
decision maker and the programmer. His documentation is 
necessarily the most broad in scope as well as variation of 
detail. It must provide intimate knowledge of the model to 
allow selection, sufficient detail to answer questions from 
the decision maker and ask questions of the programmer. It 
must provide the key to the inner workings of the model. 

Programmer documentation allows the programmer to 
maintain and modify the model and answer the questions of 
the analyst. It allows the programmer to maintain 
efficiency of operation and insure execution in accordance 
with the dictates of the design. It must clearly indicate 
how the computer interacts the inputs and programming code 
to produce the outputs. 



D. A CONCEPTUAL THEORY OF MODEL DOCUMENTATION 



Current texts that describe the documentation process 
usually are texts on the subject of computer systems. The 
guidance provided, though attempting to be general, is 
oriented toward documentation of programs that are bulk 
processors of systematic information, i.e. financial 

records, administrative records etc. Most emphasis is 
placed on how to use the available program to process the 
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data available, minimal emphasis is directed toward how the 
processing is accomplished, because the processing is not 
complex but routine and easily understood. For an example 
see Ref. 33. 

In the field of combat modeling little guidance is 
available in the form of textbooks describing documentation 
methodologies. In order to maximize detailed understanding 
as well as transferability of combat models a new concept of 
documentation is proposed and set forth in this section. 

To fulfill the requirements discussed above the 
documentation must provide the complete understanding of how 
the model was created, underlying assumptions, explicit 
description of the formulations and numerical methods 
employed and detailed description of the mechanics of the 
program code and its execution. Most current documentation 
described as a user's manual (e.g. see Ref. 22), executive 
summary or by some similar title is sufficient to fulfill 
the proposed non- technical documentation that follows. 
Programmer's documentation is widely described in the 
literature of computer systems and needs just a few 
additions in respect to combat models. Assuming that the 
documentation is provided as described in the literature, 
minimal modification is required. Primarily this is a 
charting and cataloging procedure to allow ease of tracing 
variables through the program and understanding the linking 
of subprograms to the main program. Current documentation 
requirements stress flowcharting. Flowcharting is 

comparable to electronic schematics and allows one to see 
the logical flow of the process being programmed. In the 
coding step this logical flow is translated into a 
mechanical flow of variables through mathematical and 
logical formulations and subprograms. A charting procedure 
will be like a blueprint that shows the physical connection 
of the main and subprograms. The catalog will explicitly 
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state how variables are interacted and passed between 
subprograms. 

For the analyst's documentation a new concept is 
proposed. Rather than describing the modal in the 
traditional manner, i.e. from the context of justification, 
the analyst's documentation should be presented from the 
context of discovery. [2] Using the traditional concept the 
model is documented by stating the assumptions of premises 
which determine the outputs of the model, showing the final 
mathematical formulae that represent the abstractions of the 
relavent characteristics of the modeled process, discussing 
the inputs required to support the given formulae, and 
listing the outputs that can be obtained from the model and 
inputs. 

This is not the manner in which the model was formulated 
and such documentation in the context of justification does 
not enlighten the analyst in terms of how the designer 
arrived at this particular abstraction of the combat 
process. One must conclude that this type of documentation 
is not of great help to the inexperienced analyst if he is 
attempting to understand the essence of a combat model. If 
the analyst is to gain a intimate knowledge of the 
functioning of a model, he must be able to ascertain how the 
model came to be. This provides him two invaluable 
insights. First, he gains an insight into the abstraction 
process of the designer. It is a glimpse of the reasoning 
procsess by which the designer cut through the myriad of 
details to deduce the fundamental variables that allow a 
model to approximate the complexity of a given combat 
process without having to model each of the multitudinous 
factors that compose the actual process. Second, he gains 
knowledge of the factors that were considered as 
representative of the combat process but were discarded 
because they seemed not to be necessary predictors. This 
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step in itself is worth the cost of providing this type of 
documentation. What it provides is a historic record of 
factors and related assumptions considered and reasons why 
they were ultimately rejected or accepted. The information 
provided must be succinct and allow understanding of the 
abstraction process without introducing voluminous detail 
and unnecessary costs. This will be an aid not only for 
the follow-on analyst but all those that come later. It 
will preclude subsequent analysts from expending time and 
money to determine why particular factors are or are not 
modeled. Hence, if they wish to modify or elaborate upon a 
combat model and information about previously considered and 
rejected alternatives is available, the expenditure of 
resources to reinvestigate a particular alternative and gain 
only duplicate information will be precluded. 



S. REQUIRED LEVELS OF DOCUMENTATION 



All the 
documentation 
documentation 



evidence gathered indicates three levels 
for models are required. These levels 
are: 



of 

of 



(1) decision-maker level, (facilitates communication 
between the decision maker and analyst) ; 

(2) analyst level (fosters communication between analyst 
and decision maker and enhances working relationship with 
the programmer) ; 

(3) programmer level (provides detailed understanding of 
program code and facilitates communication with the 
analyst) . 
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Figure 1 - PROPOSED HIERARCHY OF DOCUMENTATION 
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First, the decision maker who uses the model as 
to his judgement requires a non- technical description 
model. Non- technical not because the decision maker 
comprehend the technical details but because the exig 
of the organization preclude him from devoting the nec 
time and effort. Second, the analyst that exercise 
model, prepares the analysis, and assists the decision 
requires a technical description of the model. This e 
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interpret the outputs of the model. Third, the prog 
needs detailed documentation that explains the mechan 
the program. This documentation provides the mea 
troubleshoot problems encountered during routine runn 
the model and implement future modifications. The 
of activity and their required documentation are sh 
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F. DECISION MAKER'S N ON-TEC HNI CAL DOCUMENTATION 



A non-technical reference manual for use by decision 
makers should provide sufficient information to determine 
general applicability of the model. It should include a 
general description of how the model operates and the major 
components, the specific purpose for which the model was 
originally designed and the known limitations should be 
stated. This will allow the decision maker to assertain the 
overall ability of this particular model to assist him in 
the problem at hand. The manual should describe the data 
requirements, available outputs, and any options provided by 
the model. This facilitates the initial planning process 
because it provides a basis for estimating the expected time 
to gather the input data and what to expect in the form of 
outputs. Physical limitations, such as computers for which 
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usable versions of the model are available, the programming 
language used, and typical running times for the model, are 
necessary contents of the non-technical manual. The time of 
such senior managers is limited and valuable. Only that 
information necessary to provide a general description and 
basis for meaningful discussions between the decision maker 
and the analyst should be included. Any details required 
will be provided by the analyst, who will conduct, the study, 
during planning, progress, and review sessions. Yet the 
documentation must make the model sufficiently transparent 
so that the decision maker understands what is happening and 
finds the model credible for the problem at hand. 



G. ANALYST'S CONCEPTUAL-TECHNICAL DOCUMENTATION 



The analyst's conceptual-technical reference manual must 
necessarily contain detailed information on all aspects of 
the model. This is the document that determines the 

overall worth of the model as an analytical tool. If 
sufficient detail is contained then the analyst can become 
intimately familiar with the model, so that any aberation 
encountered during operation of the model can be understood 
by him and explained to the decision maker. To maximize its 
value to an analyst, it should be written by the analyst or 
analysts that design the model. Their professional 
experience will guide them in providing the kind of 
information about the model that they would want if they 
were using a model designed by someone else. 

The size of the analyst's reference manual will be a 
function of the complexity of the model. To be useful and 
credible a model must be transparent and transferable. To 
be transparent and transferable all necessary details of the 
model must be provided. Information on the data base used 
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in the model, input requirements and format as well as 
output format and options must be detailed. All that can be 
assumed is that the analyst is knowledgeable in the use of 
the tools of his profession. In preparing the analyst's 
manual no prior knowleige of the model can be assumed. As 
Morris £2] recommended, the obvious should be written down. 
All constraints and limitations must be described in detail 
as well as assumptions used, logic flow and 
interactions. Sufficient technical detail to allow the 
analyst to manually trace inputs through the algorithms is 
necessary. Mathematical, statistical, ani numerical 

methods incorporated in the model should be described 
including any new or unique applications. Any constraints 
which will affect the accuracy of the model must be 
identified. Obvious pitfalls must be stated; they are only 
obvious to the developer and in complex models without 
documentation they can even be forgotten by the 

developer. The physical processes simulated must be 

described including an explanation and rationalization of 
the techniques used. Each variable and the entity it 
represents must be clearly stated. Lastly, sufficient 
instructions describing how to set up and use the model and 
flow charts keyed to the program instructions should be 
provided. Equally important is a system that keys the 
description of each mathematical formulation in the manual 
to the appropriate section and lines of the program 

code. This will facilitate the location of the code, when 
the analyst wants to modify the model. 



These are the basic requirements for the contents of 
documents to be prepared by the designer of a model. Each 
time a modification to the model is made, the changed 
program instruction, reasons why the change was necessary, 
how and what is affected in the model, and the 

rationalization for the particular modification should be 
documented and incorporated into the original 
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documentation. This aspect of documentation is critical 
because analyst and programmers have been known to make 
small insignificant changes to programs without documenting 
them. Two results can come about because of this. Over 
time these minor undocumented changes accumulate, then one 
day those who implemented the changes transfer or depart. 
The newly arrived analyst now has a model which does not 
quite fit his documentation. If enough time has passed 
even those who made the change will not completely remember 
it when they are questioned or they are no longer 
available. Even more disheartening is that in failing to 
document, a critical inspection of the effect of the change 
on other parts of the model is not made. Unbeknownst to the 
individual the modification causes an effect elsewhere in 
the model that is not readily ■ discernable at the 
time. Utimately, another modification is made and the model 
malfunctions or an anomalous output occurs. If undocumented 
modifications have been made and forgotten it may be 
impossible to trace the cause of the anomaly. If it can be 
traced, the cost of trouble shooting and correcting the 
model will be greater than the cost of documenting the 
modifications at the time of their addition. Current 
studies will be delayed with their attendant costs and the 
credibility of the modeling community will suffer. The 
decision maker justifiably becomes skeptical when the 
analyst cannot explain why the results of a model are not 
compatible with the decision maker's intuitive 

expectations. If a model is to aid in the decision making 
process its outputs must be explainable and understandable 
to the decision maker. If the analyst has not designed the 
model then his only source of the necessary information is 
the documentation supplied with the model. 
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H. PROGRAMMER'S TECHNICAL DOCO MENTATION 



The final level of required documentation is the 

detailed information required by the programmer for the 
operation and maintenance of the model. This is the 

documentation that is discussed in great detail in all 
standard texts, while methods of documenting the conceptual 
basis of models is almost entirely foregone. The manual is 
necessarily prepared by the model programmer in conjunction 
with the analyst who has designed the model. It must 
contain descriptions of each global variable and in which 
subprograms the variable is found. Descriptions of the 
functions performed by the program, data flow charts, 
function flow charts, approximations and numerical 
procedures used are also contained in this 
documentation. Likewise, any implicit assumptions made by 
the programmer during the coding process to facilitate 
computation must be documented. 

Often, though the analyst designer has structured the 
model to handle the general case, during implementation of 
the model on the computer some loss of generalization 
occurs. In the search for programming efficiency the 
computer programmer may code a combat process so that the 
model works for only those circumstance explicitly stated by 
the designer. Whereas the designer believes he has a 
general model the programmer has reduced the generality of 
the model. Unless this is documented by the programmer, in 
subsequent use of the model the fact that the model was 
programmed in such a manner may be forgotten. In this case 
the computer routines embody more assumptions than exist in 
the formal model designed by the analyst. It is incumbent 
upon the programmer to bring this fact to the attention of 
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the analyst. He inturn must include this information in his 
documentation. Dnly he can conceptualize in terms of real 
world factors the impact of the programmer's assumptions on 
the inputs and outputs. 



Further contents of the detailed programming ma,nual must 
indicate primary and secondary storage requirements, error 
detection and recovery procedures, and instructions for 
initiation and termination of program operation. These 
pieces of information are required if other programmers are 
to be anle to understand and operate the model. If a model 
is transferred this information is necessary to implement 
the model on a computer of different design. It provides 
the receiving programmer the necessary information to 
prepare system operating programs and procedures so that the 
model can be exercised by the receiving agency without 
inadvertently making changes to the logic of the 

model. This is especially critical for complex 
models. With such models attempts to restructure programs 
to overcome incompatibilities in storage or other machine 
requirements may introduce changes to the fundamental logic 
of the model unbeknownst to either the programmer or the 
analyst. "Without adequate documentation such restructuring 
must be accomplished with only luck to guide the way and as 
complexity of the model increases luck is quickly 
depleted . 



When the original programmer completes the programming 
(coding) of the model the model must be verified and 
validated in conjuntion with the analyst designer. When 
both are satisfied that the model is operating as designed a 
listing of the compiled assembled program should be 

made. This is then supplemented by a set of working notes 
on program operations, set up of the deck to exercise the 
model, special programming features and identification of 
potential problem areas and suggested solutions. 
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The compilation of the above documentation into a 
detailed programmer's manual will provide a sound basis for 
review and analysis of the model's capabilities and for 
maintenance of the model's logic; it will facilitate 
transfer and insure transparency. Furthermore, each time 
the programmer makes a modification to the program it must 
be documented and incorporated into the programmer's 
manual. Concurrently, it must be brought to the attention 
of the analyst to relate the programming change to the 
combat process modeled and update the analyst's and decision 
maker's manual. This coordination between the programmer 
and the analyst must always occur if both are to remain 
fully cognizant of what as well as how the model simulates 
the real world. Only through such documentation can a model 
be transferred to a new programmer and analyst team and be 
knowledgeably used. 



I. INLINE DOCUMENTATION, A MAP THRU THE MAZE 

A necessary adjunct to this proposed documentation 
package is documentation within the program. Such 
documentation should be in the form of comment cards that 
briefly describe the main program, the major sub-models, 
subroutine and each function subprogram. They should be 
inserted at appropriate locations so that logic of the 
program is readily apparent. Nothing is more frustrating 
to an analyst that is attempting to gain a quick basic 
understanding of a model than a series of calls to 
subroutines which inturn call other subroutines and or 
function subprograms. In a complex modal this quickly hides 
the logic of the model from the analyst. To pierce this 
shroud the analyst must devote time that could better be 
used elsewhere. If the exigencies of the situation demand a 
rapid response and no other model is readily available this 



64 



encourages the analyst to use the model as the proverbial 
'•black box'*. This neither adds to transparency nor does it 
enhance the reputation of the analyst or operations 
research . 



Most current programming texts recommend using comment 
cards to enhance understanding of programs. In complex 
models this is not a nicety but a necessity. Time devoted 
to this effort initially not only enhances the use of a 
complex model but reduces recurring costs. Professionalism 
demands that a model should not be used without 

understanding how it functions or at least believing one 
understands how it functions. The realities of military 
personnel policies dictate that military analysts will 
rotate rapidly through assignments. If a program is not 
documented internally this self-educating step occurs each 
time a new analyst uses a particular model. Given a complex 
model and normal personnel turnover a sizeable cost is 
incurred. With adequate internal documentation this cost 
will occur only once. Because of personnel rotation models 
not currently having such internal documentation should have 
it added. Though it will detract an analyst for an 
additional amount of time initially, over the longer term a 
savings will be realized. Each subsequent analyst will not 
have to start from scratch when trying to gain an 
understanding of how the model functions; understanding will 
be facilitated by internal documentation via comment cards. 



J. UPDATING A MODEL'S DOCUMENTATION 

Changes to models occur over extended periods of time. 
Obviously, at the time changes are instituted the process 
through which they were developed, why and how they were 
entered into the model and all assumptions used must be 
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annotated in all appropriate documentation at the agency 
instituting the change. Some of these changes will be 
extremely large and will justify formal changes being 
printed and distributed for all affected 

documentation. Some changes will be small; formal printing 
of individual changes would not be justified. These would 
have to be accumulated and changes printed, when 

economically practical. The management of the process and 
the determination of whan printing and distribution of 
changes is economically practical are beyond the scope of 
the thesis. They are important subjects and should be 

examined in some future thesis. 



K. SUMMARY 



Since the analysts and programmers that develop a 
particular model are not always present or available when 
the model is exercised, especially if it is transferred to 
another agency, it is their responsibility to detail their 
assumptions, simplifications and methodologies and provide 
evidence that the rationale behind their approach will 
produce results usable in the real-world environment as 
viewed by the ultimate decision maker. Analysts and 
programmers that create a model must provide documentation 
that establishes the issues examined by the model, 
underlying the objectives and assumptions, the usability and 
usefulness of the model. Only with such documentation can 
the analyst or analysts assisting a decision maker in the 
resolution of a particular problem conclude that the use of 
a specific model is appropriate. The real "user" of a 
model is the decision maker. The analyst must recognize 
in assisting the decision maker he is a model designer and 
model manipulator. The officer analyst must gain detailed 
knowledge of any model he uses in support of an analysis. 
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To acquire such knowledge a model must be adequately 
documented. In combat modeling there are three levels of 

participation each with unique documentation requirements. 
These levels are: 

■ Decision-maker level; 

■ Analyst level; 

■ Programmer level. 

To enhance the ability of the analyst to fully 
understand the model his documentation should be presented 
from the context of discovery rather than the traditional 
context of justification. This will allow the analyst to 
know the alternatives considered and rejected in structuring 
a model as well as giving him greater insights into the 

underlying hypothesis of a model. 

The types of documentation required are: 

■ Decision Maker's Non-technical Documentation; 

■ Analyst's Conceptual-technical Documentation; 

■ Programmer's Technical Documentation; 

■ Inline Documentation. 

Since model modification and improvement is a continuing 
process the preparation of formal changes must be 

economically practical and carefully managed. This area of 
model documentation is suitable for a subsequent thesis. 
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VII. CO NCLUSIO NS f RECOMMENDATIONS^ AND FINAL REMARKS 



A. SUMMARY OF PROBLEMS DISCUSSED 

Since 1973 the impetus in the Army modeling community 
has been to develop scenarios and models that can be used 
throughout the Army community for decisions on material 
requirements and force development. The ultimata purpose is 
to bring consistancy into the decision making 
process. Inherent in this goal is not only the need for 
standard agreed upon scenarios but for a repertory of models 
to be used by all interested agencies examining a particular 
facet of the Army. This 3oes not necessarily mean that only 
one model should be used for a particular 
investigation. Because of the assumptions necessary to 
develop any model, given the necessary time and money it is 
best to exercise more than one model in order to get an 
indication of how the assumptions affect the outputs of each 
of the models. What is fundamental is the need for all 
agencies examining a given situation to be able to 

understand each other's models; this will provide the basis 
for intelligent discussion of the pros and cons of equipment 
and force requirements. 

As operations research and system analysis have gained 
acceptance by DOD and Department of the Army, more combat 
models of various levels of military operations have been 
created and used. Increased use of models resulted in 

increased levels of complexity. Complexity in turn caused 
new models to be created when potential users could not 
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sufficiently understand and use existing models given 
available documentation. This unnecessary proliferation 
resulted in unexplainable conflicting results which caused 
the credibility of combat modeling to be questioned. If 
this situation is to be resolved unnecessary proliferation 
must be checked and mutual understanding enhanced. To 
achieve this goal models must be easily transferable between 
investigating agencies. Fundamental to this ability is 
complete and up to date documentation of the model to be 
reviewed or exercised. This documentation will allow the 
potential user to analyze the model and determine its 
suitability for a given project. 

Concomitant with the need to easily transfer models is 
the need to fully understand the machination by which a 
given set of input data is acted upon to produce 
outputs. If outputs of a computer model are to be useful 
they must be credible. Conflicting outputs of military 

models are consistently challenged by some government agency 
of the DOD, the Congress, the Executive Office or some other 
branch of the government. CJnless these challenges are 
answered and conflicts explained, the credibility of combat 
modeling will continue to suffer. Some criticism is needed 
to purify combat modeling and identify errors that 
inevitably will appear, some eminates from those advocating 
a competitive model or concept and some of it is from those 
that criticize the validity and usefulness of combat 

modeling itself. At times inadequate documentation has 
precluded the explanation of conflicting results by the 
analyst and added to the criticism. 

The Comptroller General [15] found that existing models 
have been used by other than the designer without thoroughly 
understanding their implications and limitations. At times 
this has resulted in erroneous conclusions being drawn and 
decisions made based on these conclusions. Subsequently, 
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the errors are surfaced with a loss not only in dollars 
expended in pursuit of undesirable projects but in further 
loss of credibility for theatre-level combat modeling. 

Adequate documentation will help alleviate some of the 
adverse publicity and loss of credibility that theatre-level 
combat modeling has experienced in the past. To a great 
extent this has stemmed from unexplainable contradictory 
results using models purporting to represent the same 
process. To allay the criticism of models and their 
outputs and to enable both the analyst that exercises the 
model and the decision maker that uses the outputs to aid in 
the decision making process the model must be 

understood. Understanding can be enhanced through proper 
documentation. Adequate documentation of the form proposed 
will facilitate communication between the decision maker and 
analyst as well as between the analyst and the 

programmer. Unless these communication links are 
established misunderstanding of combat models will persist 
and the credibility of combat models and modeling will 
suffer accordingly. 



B. CONCLUSIONS AND RECOMMENDATIONS 



The conclusions drawn from this research are: 

■ The decision maker is the "user" of a model. 

■ To be of value a model must be accepted by the 
decision maker. 

■ The analyst must relate the abstractions of the 
model to the actual combat process. 

■ The analyst is a model manipulator. 

■ The analyst must understand and explain the 
methodology and structure of a model. 

■ Understanding is deterred by complexity. 
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■ Models are becoming more complex. 

■ Increased levels of complexity result in 
diminished transparency. 

■ Inability to understand and use existing model 
causes development of redundant models. 

• Unnecessary proliferation causes conflicting 
results to be produced by similar models. 

■ Inability to explain conflicting results is the 
basic cause of a lack of credibility. 

Proliferation can be reduced through model 
sharing. Sharing a model requires: 

■ ability to use it with minimal conversion; 

■ adequate facilities to run the model; 

■ sufficient competent analysts and programmers; 

■ adequate documentaton; 

■ formalized arrangements for cost sharing and 
coordination. 

Prior to transferring a model it must be analyzed for 
applicability and appropriateness. Such an analysis should 
consider : 

■ inputs and outputs; 

■ basic causal relationships; 
m model logic and structure; 

■ available data and required data; 

■ time and resources required. 

Many existing Combat models are characterized by: 

■ submodel inadequacy; 

■ key variables excluded; 

■ inherent inaccuracies; 

■ computational difficulties; 

■ inconsistent assumptions; 

■ inappropriate assumptions; 

■ inadequate output. 
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The degree to which meaningful model evaluation can be 
accomplished is significantly influenced by available 
documentation. Documentation is key to the understanding 
of complex models. The research conducted indicated: 

■ adequate documentation is a necessary if a 

model is to be used by other than the 

originator . 

■ Organization exigencies deter adequate 

documentation. 

■ Documentation of combat models is generally 
inadequate. 

m If a model is poorly documented it may be more 
economical to build a new one than share an 
existing model. 

■ Efforts are being made to improve documentation. 



The author's experience with and research of combat 
model documentation indicates that there are three levels of 
interaction with combat models. These levels have unique 
and common requirements for documentation. To satisfy these 
requirements the author envisions three levels of 
documentation: 

■ DECISION MAKER'S NON -TECHN ICAL 

■ ANALYST'S CONCEPTUAL-TECHNICAL 

■ PROGRAMMER'S TECHNICAL 



The decision maker's and the programmer's documentation 
must provide the information listed in chapter six. It can 
be presented in the traditional manner using techniques 
contained in most computer system management texts. 

However, to function as the link between the decision maker 
and the programmer and to understand the nuances of the 
model, the analyst needs documentation that provides greater 
insights than possible with the current available 

documentation. These insights will be provided if the 
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analyst's documentation is presented from the context of 
discovery rather than the traditional context of 
justification. 
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C. FINAL 3EMAHKS 



Acceptance or rejection of an expository thesis in 
matters such as documentation often depends on the skill of 
the pleader and the mood of the audience. Staring at the 
same set of evidence the parties to the debate can come to 
sharply different conclusions, since their preconceived 
notions may lead them to select and interpret the evidence 
in different ways. Even though one may initially find it 
difficult to believe that there are ways to acquire adequate 
documentation not yet tried by analysts or advocated by 
agencies researching the problem, the very complexity and 
pervasiveness of the problem suggests the possibility of 
combining the various proposals in different ways so that 
some combination will produce the desired goal. The 
proposal presented is but one possible means to achieve the 
desired end. Of even more importance is the fact that some 
methodology must be adopted to correct this lack of adequate 
documentation. With regard to theatre-level combat models, 
the problem has persistei for almost twenty years. Not only 
has it made it near impossible to easily transfer models 
between interested agencies but it has prevented military 
analysts from gaining full knowledge of the models they 
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use . 



The conflicting opinions and evaluations unmasked during 
this research confirm what is intuitively obvious: many of 

the historical judgements and decisions concerning 

operations research in general and theatre- level combat 

« 

modeling in particular are based on subjective values as 
well as objective facts. Unlike the natural scientist or 
the analyst using a simulation, the researcher examining the 
process through which combat modeling has evolved, cannot 
reproduce the events aQd by experimentally altering the 
ingredients, change the result. The development of combat 
modeling is well documented; yet controversies have 
developed despite the voluminous sources. Analysts disagree 
not because one may be more knowledgeable about the subject 
than another, but because each weighs and evaluates 
differently those facts of which both have knowledge. There 
is little dispute about the details of what has happened in 
the development of th eatre- level combat models but there is 
intense disagreement on the significance of past events and 
how to proceed in the future. The analyst has no fixed 
point from which to observe the stream of events concerning 
the development of theatre-level combat models. Analysts 
are borne along by the current and their interpretation of 
what has occurred is influenced by their view of where the 
stream seems to be headed and whether the apparent 

destination appears to be good or bad for the enhancement 
and development of the OR profession. 

Although theatre-level combat modeling attempts to be 
scientific in its methods, it is rarely so in its 

outputs. Outputs are interwoven with subjective judgements, 
either through their interpretation or by way of the inputs 
that were instrumental to producing the outputs or in the 
very construction of the model itself. The relativity of 
subjective judgement, while discouraging, need not be 
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debilitating to theatre- level combat modeling. Because 
these models are the only means of examining force structure 
questions vital to national security, analysts are 
confronted with the continuing task of rethinking the basic 
structures of these models. Past models can furnish us a 
vast reservoir of experience in theatre- level combat 
modeling which can be exploited to further this aspect of 
operations research. However, this reservoir can be 
effectively used for the enhancement of the profession only 
if what has been accomplished is adequately documented so 
that others may correctly use models previously 

developed. In this manner, even though no model can fully 
treat all the intricacies of the combat process, the analyst 
can enrich the profession through the continuing effort to 
better model the process fully using all the knowledge that 
has come before. 
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understood; any new undocumented factors uncovered in its 
examination should be formally noted and made a permanent 
part of the official documentation. In this manner past 
omissions will be corrected, the scientific method will be 
invigorated and the standing of the Operations Research 
profession enhanced. Subsequent results will then be more 
fully explainable and posses greater credibility and even if 
the conclusions cannot be final, because of the impalpable 
nature of the subject, the techniques of theatre-level 
combat modeling will be enriched by the process. 



76 



LIST OF REFERENCES 



1. J. Honig, et al, R ev iew of Selected Army Models, p. 

4-9, 0. S. Army Model Review Committee, May, 1971. 

2. 5*. T. Morris, " On The Art Of Modelling", Manag emen t 

Science, v. 13, p. B-737 - B-717, 1967. 

3. R. Huber (Editor), Op era t ionanaly tische S pi el e fur d ie 

V erte r dig ung , p. 139 - 189, R. Oldenbourg 7erlag, 

Munchen, 1979. 

4. F. McHugh, Punde mentals of War G am ing, p. 2-1 - 2-58, 

Naval War College, march, 1966. 

5. Joint War Gaming Agency, J oi nt War G a,m ing Manual, p. 

31 - 103, Organization of the Joinr Chiefs of- Staff, 

1969. 

6. A. Enthoven and K. Smith, How Much Is .Enough, p. 31 - 
116, Harper and Row, 1972. 

7. S. Bonder, An Overview of Land Bat tl e Mod elling in th e 
0. S ir p. 73-88, Proceedings of the 13th Annual U. 
S. Army Operations Research Symposium, Ft. Lee, Va., 
1974. 

8. R. Huber, A System A nal y st 1 s View On Force Structure 
Pl ann ing, p. 30-41, paper presented at the Naval 
Postgraduate School, July, 1979. 

9. R. Murray, The Analyti cal Profes si on^ Its S tandard s 
and Its Future, p. 1-6, keynote address 40th Military 
Operations Research Society Meeting, Dec, 1977. 

10. M. Shubick and G. Brewer, Models^ Simulatio n s, a nd 



77 



Hay, 1972 



Ga mes - A Survey, p. 4-71, Rand Corp. , 

11. G. Brown, Jr., "Managing Analysis", Naval War Colle ge 
Rev iew , v. 32, p. 22 - 28, May-June, 1979. 

12. R. Huber (Editor), Military Strategy and Tactics, p. 
3-19, Plenum Press, 1975. 

13. E. Quade (Editor) , An aly sis for Mil itary De cision, p. 

300-316, Rand Corp., 1964. 

14. S. Gass, " Evaluation of Complex Models", Com pute rs 

a n d Op e rat ions Research, v. 4, p . 27-35, Pergamon 

Press, 1977. 

15. General Accounting Office, Report to Congress - 
Im provem ent Nee ded in Docume nting C om puter sys tems . p. 
1 - 28, Comptroller General of the U.S., Oct., 1974. 

16. M. Shubik, Games for S oci ety . Bus ines s and War, p. 181 

-319, Elsevier, 1975. 

17. M. Shubik, The Uses and Methods of G ami ng , p . 49 - 
116, Elsevier, 1975. 

18. D. Hardison, The Comp lexity Crisis, p. 3 - 18, keynot 

address 15th Annual U.S. Army Operations Research 
Sysposium, 1978. 

19. A. Cordesman (Editor), Developments in Th eater Lev el 

Wa r Games . p. 135 - 236, unpublished pgpers for C-5 
Working Group of 35th Military Operations Research 
Symposium, 1975. 

20. R. Farrell, Mode l Choice, p. 1 - 23, Appendix L to 

notes for "Topics in Military Operations Research" 
presented by V ecxor Research Inc., Ft. Belvoir, Va. 
1971. 

21. T. Dupuy, Numb ers, Predictions And W ar , p. 3 - 18, 

Bobbs-Merrill Co., 1979. 



78 



22 



2. Kerlin and R. Cols, ATL AS : A Tac t i c al , Logist ica l , 

and Air Simu lation ; Documentation and Use r 1 s Guide, p. 
3 - 99, Research Analysis Corp. RAC-TP-338, 1969. 

23. D. Moder and D. Edwards, Compute ri zed Qu iet Ga m e (Vo l. 
11 . Mod els) , p. 7 - 58, Research Analysis Corp., 1967. 

24. General Accounting Office, Report to Congr ess - 
Mo dels, Data and Warp A Critigue of the Found ation f or 
De fe nse Decision, p. 50-78, Comptroller General of 
the U.S., July, 1979. 

25. Methodology and Resources Directorate, Mod if i c at ion s 

to atlas TPzZ 4-3, p. 1 - 6, U.S. Army Concepts 

Analysis Agency, July, 1974. 

26. S. Shu pact. An E xami na tion of the Attrition Pr ocess in 
the I nsti tute for Defense Analysis Ground- Air Model , 
p. 9 - 19, M.S. Thesis, Naval Postgraduate School, 
Monterey, 1979. 

27. E. Kelleher, Simulation of the Ta ct i cal E mp l oym ent of 

Field Artille ry, p. 10 - 70, M.S. Thesis, Naval 

Postgraduate School, Monterey, 1977. 

28. E. Hagewood and W. Wallace, Simulation of Ta ctic al 

Alternative Responses , p. 10 - 80, M.S. Thesis, Naval 
Postgraduate School, Monterey, 1979. 

29. W. Caldwell and W. Miers, An Air -to -Ground a nd 

Gr ound-to-Ai r Comb ined Ar ms Combat Simulat ion, p. 10 - 
100, M.S. Thesis, Naval Postgraduate School, 

Monterey, 1979. 

30. D. Emerson, New TAGS T heatre - Air -G round Warfare Mod el: 
A Des cripti on and Operat ing Instru ctio ns, p. 1-92, 
Rand Corp., Sept., 1974. 

31. Command and Control Technical Center, Vec to r- 2 System 
for Simu lation of Thea ter - leve l Comba t , TM x x x — 7_9 , p. 



79 



5 - 294, Vector Research Inc., 1978. 

32. R. Campbell, Tank Battle Simulat io n Model, p. 1 - 23, 

Consolidated Analysis Center, Inc., June, 1979. 

33. S. Mixon, Handbook of Dat a Processing Admini st r ati on, 

Ope ration s and Procc edures, p. 271 - 328, Amacon, 

1976. 

34. E. Quade and W. Boucher (Editors) , S yst ems An alys is 
an d Polic y Planning, p. 21 1 - 254, Elsevier, 196 8. 

35. D. Me Cracken, A Ouide To Fo rtran IV Pro gramming , p. 

225 - 229, Wiley , 1972. 



80 



INITIAL DISTRIBUTION LIST 



No. 



1. Defense Documentation Center 
Cameron Station 
Alexandria, Va. 22314 

2. Library, Code 0412 

Naval Postgraduate School 
Monterey, Ca 93940 

3. Department Chairman, Code 55 
Department of Operations Research 
Naval Postgraduate School 
Monterey, Ca 93940 

4. Professor J. G. Taylor, Code 55 Tw 
Department of Operations Research 
Naval Postgraduate School 
Monterey, Ca 93940 

5. LTC R. S. Miller, USA, Code 55 Mu 
Department of Operations Research 
Naval Postgraduate School 
Monterey, Ca 93940 

6. LTC R. W. Szymczak, USA 
9129 Maywood Lane 
Fairfax, Va. 22031 



Copies 

2 

2 

1 

5 

4 



81 



The 

S9S 

c.I 



Thesis 

S99 

c.I 



Sz ymc2ak 18713 4 

Transferabi | / tv nf 
combat models- n • 

,, ** tlon * '"Posed by”''" 

1 3 A UJ'S^mentat ion, 
'^AUCtiees. V*i 

3 H s /M& 




Szymczak f 8 7 134 

Transferabi I ity Q f 
combat models: ? im i- 

nations imposed by 

documentation prac_ 



thesS99 , , 

Transferability of combat models . 




3 2768 001 01268 5 

./unv i IDDADV 



